[None][fix] harden Python native KV transfer ownership#16396
[None][fix] harden Python native KV transfer ownership#16396chienchunhung wants to merge 5 commits into
Conversation
3b5f743 to
3862973
Compare
|
/bot run --disable-fail-fast |
|
PR_Github #59340 [ run ] triggered by Bot. Commit: |
|
PR_Github #59340 [ run ] completed with state
|
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
3862973 to
6888a21
Compare
|
/bot run --disable-fail-fast |
|
PR_Github #60892 [ run ] triggered by Bot. Commit: |
|
PR_Github #60892 [ run ] completed with state
|
Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>
|
/bot run --disable-fail-fast --stage-list "A10-PyTorch-2" |
|
PR_Github #61124 [ run ] triggered by Bot. Commit: |
|
PR_Github #61124 [ run ] completed with state
|
Important
Integration draft — not ready for code review as one PR. The current branch combines several ownership layers and will be decomposed into review-sized production+test slices. Please use #16347 for the design contract; do not review this aggregate diff yet.
Summary
unpinBlocksByIdvalidationRelationship to #16347
This PR is the first implementation follow-up to #16347 and is logically stacked on it. The #16347 head branch lives in the fork, so this PR targets
main; until #16347 merges, GitHub will also show its three design-document changes in this PR.Applicability
transceiver_runtime="PYTHON"selects Python-nativeKvCacheTransceiverV2kv_cache_bounce_size_mbor a second ownership flagCacheTransceiver, its data path, and its wire protocol unchangedSafety and operational impact
The normal path gains lifecycle bookkeeping and exact-set validation but no additional device arena beyond an already-active bounce configuration. Under ambiguous transport failure, the implementation deliberately retains requests, KV allocations/pins, bounce slots, and transport mappings instead of risking reuse or use-after-free. That converts a potential memory-safety failure into visible capacity pressure or a teardown veto; bounded recovery requires the follow-up lease/protocol work described in #16347.
Validation
trtllm-reviewself-review through a final clean pass with no P0-P2 findingspre-commit run --from-ref FETCH_HEAD --to-ref HEADgit diff --checkgit range-diffconfirmed all three patches were unchanged when rebased onto currentmain(d17ba0e4d)Not run locally: the full pytest selection (the environment lacks
transformers), compiled C++ unit tests, or GPU/NIXL/VMM/PP/ADP/multirank runtime tests. Those remain required before this draft is ready for merge.